The present invention relates to an improved eye tracking device used to determine the visual axis and point of regard of a user, and more particularly to an improved eye tracking device which utilizes light detection devices curved around the optical axis of the eye so as to always be coincident with source-emitted light reflected from the fovea, and conversion circuitry for determining the position of the fovea reflected light and computing the visual axis and point of regard at a very rapid rate.
The concept of retinal-reflected light as utilized in the invention is shown in FIGS. 1 and 2. This concept is well known in the art and is discussed only briefly here. A full discourse on the concept is given in J. Krauskopf, Measurements Of Light Reflected From The Retina, National Research Council, Pub. 1272, pages 149-170 (1966). FIG. 1 shows the various parts of the eye 20. The outer surface of the eye 20 includes the cornea 22, the sclera 24, the iris 26, and the pupil 28. The inner parts of the eye include the lens 30, the retina 32, and the fovea 34. The position of the fovea 34 generally corresponds with the optical axis 36 of the eye 20. When collimated light is shined directly onto the eye 20 it will be reflected by the various external and internal parts of the eye. The brightest reflection is always that which exits the pupil 28, since light shined onto the pupil 28 passes through it, is focussed onto the retina 32, and is then reflected by the retina 32, and more particularly by the fovea 34. Since the position of the fovea 34 corresponds to the optical axis 36, the visual axis and point of regard can be computed if one can accurately and quickly determine the position of the fovea-reflected light.
FIG. 2 graphically illustrates the concept of eye reflected light and the greater intensity reflection by the fovea 34. The Y axis represents the intensity of light reflected from the eye. The X axis represents the parts of the eye shown in FIG. 1. Points 1 and 2 on the graph correspond to light reflected from the sclera 24. Points 3 and 5 correspond to light reflected from the iris 26. Point 4 corresponds to the brightest reflected light, that from the pupil 28 (which is from the fovea 34), as expected.
The frequency of light used for reflection off the eye and retina plays an important role in determining the visual axis and point of regard. As is known in the art, only light in or near the upper limit of the visible spectrum range is capable of being reflected by the eye and retina. The use of infrared light, then, is unacceptable, since the eye absorbs all infrared light and therefore no reflection will occur. On the other hand, plainly visible light is not acceptable either, since shining visible light into a user's eye will of course result in a blinding effect, and thus no useful point of regard data can be obtained. Therefore, it is necessary to use light which falls in or near the visible spectrum but will still be transparent to the eye's (specifically, the fovea's) sensitivity. Near infrared light is the type most suitable for these purposes, since it falls near the upper limit of the visible spectrum but will not appear visible to a user, since it is outside foveal sensitivity. FIG. 3 graphically depicts this concept. The visible spectrum generally falls between light wavelengths of 380 to 770 nanometers (nm), and the near infrared light range starts at about 770 nm. Peak foveal sensitivity is around 555 nm. Light most suitable for foveal reflection, therefore, is that light which begins at the upper limit of the visible spectrum and extends into the near infrared light spectrum, for example, light in the range of 675 to 1000 nm.
Prior art eye tracking devices have relied on various eye characteristic concepts to determine the eye's position. FIGS. 4A and 4B illustrate two of these concepts. FIG. 4A is the basis for corneal reflex (oculometer) eye tracking devices. An enlargement of the eye 20 is shown. The optical axis 36 is aligned with the center of the pupil. Light 40 from a source within the oculometer (not shown) is reflected by the cornea 22. A video camera (not shown) looking at the eye 20 finds the brightest spot on the eye, which is the reflection from the cornea 22, and also finds the darkest spot on the eye 20, which is the pupil 28. A video processing computer (not shown) computes the location of the center of the pupil 28 from the picture provided by the video camera. The displacement of the corneal reflection from the center of the pupil is K sin .theta., which is proportional to the angular direction, .theta., of the eye 20. This displacement is independent of the position of the eye 20 in its socket (not shown). By determining the displacement of the corneal reflection from the center of the pupil, an approximation of the point of regard may be computed.
FIG. 4B represents a concept known as Purkinje images, which is also utilized in prior art eye detection devices. As light 40 passes through the eye, reflections occur at every interface in which there is a change in dielectric constant. There are four surfaces where these reflections occur: the front of the cornea 22, the back of the cornea 22, the front of the lens 30, and the back of the lens 30. These reflections correspond to the four Purkinje images, respectively. The fourth Purkinje image is almost the same size and is formed in almost the same plane as the first Purkinje image, although it is over 100 times less intense. If the eye rotates, these two images change their separation in space because the surfaces that form the first and fourth Purkinje images have centers of curvature that lie at different distances from the center of rotation of the eye. The physical separation between the first and fourth Purkinje images in space is a measure of the angular rotation of the eye in the direction of the rotation. By determining the distance of the separation of the first and fourth Purkinje images, an approximation of the point of regard can be calculated. Again, this requires the use of a video camera to find the Purkinje images and a video processing computer to compute the location of the Purkinje images. A typical prior art eye tracker that utilizes the Purkinje image concept is disclosed in Crane et al., U.S. Pat. No. 4,287,410.
While both of the concepts described in FIGS. 4A and 4B above have improved the field of eye detection devices, there are disadvantages to their use. Both methods require a video camera to be mounted in a fixed position relative to the eye. This usually entails use of a cumbersome helmet mounted device. Both methods also require expensive and slow video processing computer hardware to process the image from the camera. Additionally, the range of eye motion that these methods can detect is very small. Large excursions of the eye within the socket result in the reflections being removed from the camera eye.
A typical example of another type of prior art eye detector is shown in Japan Patent No. 63-210613, which discloses a glance direction detector whereby a matrix of light emitting diodes emit infrared light that is reflected by a mirror so as to reflect off the retina. The retina-reflected light is in turn reflected off another mirror and is detected by a light receiving body such that the glance direction of the eye is detected. Although providing improvements over previous prior art eye detectors, the disadvantages of such a system are at least three-fold: the use of reflecting mirrors requires a great amount of precise calibration in order for the device to function properly; the mirrors must be very large in order to reflect wide angle changes in the position of the eye, resulting in the mirrors taking up an unacceptable amount of surface area; and perhaps the greatest disadvantage is that the use of only one light receiving body on a flat surface results in non-linear eye location information being outputted, depending on the extremity of the eye's position. This is because the distance the reflected light travels from the eye to the light receiving body will vary as the eye moves in the socket. The ultimate result is that only an approximation of the actual eye position is given, which is unacceptable to those requiring accurate eye position data, or alternatively computer mapping will be required, which is prohibitively costly and time consuming.
Other prior art devices are of the type epitomized by Baldwin, U.S. Pat. No. 4,568,159. Baldwin shows a head and eye position indicator that utilizes an infrared light source reflected from the cornea of the eye to yield eye position, as well as additional coded infrared sources which are reflected and detected to determine head position. Baldwin uses an array of infrared emitters dispersed around a spherical dome upon which the visual presentation is displayed. The infrared emitters are coded such that each can be identified by its location on the dome. The reflected infrared light is transmitted via a fiber optic means to a charge coupled diode array which provides a high speed conversion into electrical impulses indicative of the relative positions of the sources within the field of view of the user, so that head and eye position may be computed. Although the invention in Baldwin provides advantages in the field of flight simulation by disclosing a head tracker and eye tracker used in tandem, it does not provide any innovation in the field of eye tracking devices alone, and thus the same problem with respect to non-linear eye position information being outputted occurs, as described above.
In addition, visual systems that employ the use of eye tracking devices are exemplified by Mallinson et al., U.S. Pat. No. 4,479,784, which discloses a visual system for providing high detail, high resolution imagery anywhere a user is looking throughout a wide field of view. The Mallinson system detects changes in the instantaneous position of the eye using a helmet-mounted oculometer system to provide a foveal view, or high detail image, at the next lookpoint. Thus, while Mallinson discloses an improved visual system, the use of a prior art eye detection device will limit the effectiveness of the system due to non-linear eye position information being outputted, as has heretofore been described.
The prior art also discloses circuitry for use in eye tracking devices. For example, Marshall et al., U.S. Pat. No. 4,387,974 discloses a circuit for calculating the position of the cornea using a dual axis infrared light detector responsive to pulsed infrared light reflected from the corneal area of the eye, such that the X and Y coordinate positions of the cornea are determined. Although this type of circuit improves over the prior art by providing enhanced position measurement accuracy and response time, there still exists a need for improved position accuracy and a faster response time than currently disclosed in the prior art.
While the prior art provides important advantages, there still exists the need for a relatively lightweight, highly accurate eye tracking device that computes the visual axis and point of regard at rates faster than that known in the prior art. Thus, what is disclosed is an improved eye tracking device, which utilizes a plurality of light emission sources and a plurality of light detection devices mounted in such a way on a head apparel device so as to always be co-incident with the axis of the eye. In this manner, the need for mirror calibration is eliminated. Another aspect of this invention that improves over the prior art is the use of conversion circuitry, such as a pyramid cascade circuit or a microprocessor and digital multiplexor, that computes the visual axis and point of regard at speeds faster than any previously disclosed eye tracker. A further advantage occurs by embedding the light emission devices and the light detection devices into a thin film of transparent polymer. By curving the thin film of transparent polymer to match the curvature of the eye, and mounting the curved thin film onto the inside surface of a head apparel device so that it extends around the inside surface and corresponds to the visual field of view, the problem of non-linear eye position information is solved. A still further advantage is the use of a mild reflector on the outside surface of the head apparel device in order to keep the amount of external light that impinges on the detection devices (i.e., light noise) at a minimum. Practical advantages upon implementation will also be apparent. The entire device may be embedded in the curved thin film, resulting in a one-piece, sturdy, solid-state eye tracker. Also, the elimination of video cameras and video processing computers results in relatively low implementation costs. All of these advantages result in highly accurate eye position information being outputted at extremely rapid rates.